133 replies to this topic

[Darian DelFord]

The Reason Clan 73% Win Rate On The Last Test Must Be Taken With A Grain of Salt!

First and foremost I am an IS Pilot. I am part of a Davion unit and as such will not be using the clan mechsin CW. With that said I have bought every package that PGI has put out and I will buy the Man of War package as well next pay check. I have a problem with the 73% number that is being thrown around. One of the problems with balance in this game is it is not based on consistency. Meaning Knee Jerk reactions to try to fix a few problem chassis, and as such there are blanket wide changes that truly destroy some mechs and builds.

What I would love to see is another IS vs Clan test using a control group. That control group being players of equal ELO (Prefer Skill base but hard to get those numbers) preferably from the higher ELO range. The reasoning is when you have data coming in from all over the place, less than reliable does not even describe it. In the matches that I participated in during the last test the IS won decidedly in all of them, it was not even a match, it was a ROFLSTOMP by the IS. My question is, what skill levels participated? More importantly, was it consistent players?

Was player A in the same mech throughout the test or did he change into a mech he does not normally play? There are so many variables you cannot base balance based off of a mere percentage which can easily be proven false.

Case in Point, the other morning did a drop just before work, went to make a cup of coffee came back and the game was 3 minutes in and thought damn. I piloted a Kit Fox Prime ECM and 2 LPL’s . By the 3 minute mark the score was 2 to 8 in the enemies favor. By the 5 minute mark it was me alone against 8 other mechs. I thought OK time to hunker down. The map was HPG I was up top so I started sniping like mad, down went an Atlas D. Had to jump down as I was taking fire up top from another ECM mech, went to the basement, ran into an ECM Kitfox with moderate damage, legged then killed him. Scuttled from the basement trying to get to the outside where I ran into an awesome LRM boat with only a MPL as a backup weapon, leg humped him and killed him. Ran into another ECM Kitfox with no damage, managed to get in behind him before he saw me and got two alphas onto a leg and legged him prior to him turning around, he died.

Then a Dual 20 Jager, then a ShadowHawk, then another Jager and finally the DireWolf whom I am assuming had Seismic Sensor as he was perfectly placed every time I tried to get behind him. I toggled my ECM yet again to try to not give away my position with my bubble as I did with the other mechs, but did not work, he backed against a wall. Then with less than 30 seconds remaining he started to move to take me out. Needless to say it came down to the last second and literally at the last second I legged him then took out his other leg and we won the match.

The moral of the story is my Kit Fox is not OP, no one can call it that. It was the player behind the mech that kept his cool knew his own mech’s weakness and strengths as well as the enemies and knew when to strike and when to run. This is why that 73% to me is not reliable. I consider myself an average player in the Kit Fox, the whole time I was doing this, I was wishing I was in my Jenner and how much easier this would have been if I were. I do not consider myself the greatest pilot, I just had a great match. But the calls of the OP’ness of my play by the other team got me to thinking about that stupid 73%.

If PGI really wants to see where the balance level is, hold another tournament but make it IS vs Clans. Or make it a weekend event or even for 24 hours. Announce it, Get more data, before any more nerfs kill the clans. I already laugh at most of them. There just needs to be a better sampling of data then a few hours of PUG match ups in which you match players against other players unevenly.

Edited by Darian DelFord, 24 September 2014 - 08:35 AM.

[Brody319]

I'd say just give us a mech in each weight class, 1 IS and 1 Clan mech as trials. Then just check the data over the next few days. if the clans are OP Newbs will do a whole lot better on average when they pilot clan mechs vs IS mechs.

Edited by Brody319, 24 September 2014 - 08:34 AM.

[Mcgral18]

A test, with controls in place, with 24 competent players, would be the best method.

Switch the team factions after whichever set, and it should get some more accurate data.

[Kain Demos]

People are just sore losers. If I have a terrible match in my Dire Whale (left behind, singled out by scouts and *****) nothing will be said.

However, a few times I've done 800-1200 damage with 5+ kills and people on the other team go to "all" after they die to call you a no-skill P2Winner in an EZ mode 'mech and clans need to be giganerfed.

You just see it a lot if someone has a great matchin a 'mech--some sore loser on the other team will cry "OP".

[Murphy7]

I understand where you are coming from, and think that there are things that can be done to get the testing where you want it to be, but I specifcally disagree with your solution/premise for two reasons:

(1) I disagree that testing in the solo queue was worthless. That game mode is one that a large number of players make use of routinely at a variety of skill levels, and the perceived balance in that area is very important to player experience of the game. It really is an entry way, and while I agree that players should be encouraged to the community TS servers and group play early and often, the perceived balance in this entry area is important. What the data from that test suggests to me is that some filter or refinement of taking into account the balance of technologies along with ELO in the matchmaker might be important in the long run.

(2) Drawing a testing pool according to an advertised or perceived "higher" ELO crowd would be deleterious to the community in the long term. We already have competitve play vs others, and chest thumping about ELO scores and who does and does not use a steering-wheel to pilot their mechs. Being able to point to the test as proposed by you as validation for some people's points of view or to invalidate others based on their exclusion from this test would just not be helpful.

I would like to see something along the lines of a group test for IS vs Clans, or even a faction unit tourney for clans vs inner sphere to see if the same level of imbalance is prevalent when communication and coordination are similar on either side. I have no idea if the balance would be exacerbated towards clans in that scenario or if the divide is closer between clans & IS in that scenario, but I suspect (and hope) that the latter is true.

[Revis Volek]

Tournament, NOOBS, Trial mechs, none of this is valid for testing.

Before we even start we need to form a testable hypothesis in order to do that we must make an educated guess that we can either prove or disprove. and not just a well screw it i think this one...no shots in the dark. Make a REAL educated gue

Just throwing test servers up and having anyone jump in weather the have an agenda or not is ok but it really isn't accurate. Its defiantly a place to start BUT WAS NOT the place PGI should have ended.

I saw no control, no measure of the testable experiments other then Win and Loss which is not always a good indicator (if matches went to time limits or was even deaths at the end or close to) and how many were alive. I never even saw a statement PGI claimed to disprove. Everyone went in with the mindset CLANS ARE OP and im sure the IS guys being bias had something to do with it.


Either way it was poorly executed, poorly researched and just a shot in the dark at something they didn't understand fully.

[Hoax415]

No a test with very few players is not a better dataset. You people do not actually understand testing and how it works and how you get robust useful data.

I guess I will try to explain even though its a waste of time with people like Mcral18 who are obviously just pushing an agenda with no basis in reality and its a double waste of time because someone else is just going to start another one of these idiotic threads with the same bad thesis in a couple of days.

Lets say you run your 24 player test. You get two teams of players who are all within 100 elo of eachother per the MM. You have them play 10 matches. 5 where team A is clan and 5 where team B is IS.

According to the logic here that is some kind of super test. Those 10 matches are much more valuable data compared to 200 matches with trial mechs, bads, bad builds, bad chassis, elo difference etc. etc.

Right?

Wrong.

Less players and less matches is not going to get you a better set of data. That's not how it works.

Will you also be controlling for maps? Just run the test on one map right? No wait what if one team is much much better on that map than the other team.

Ok how about we up the test to 16 matches, 4 matches on 4 maps each team plays as each faction twice?

Well we still have the problem of what if one team is stronger on one type of map and you run 3 of those type. Or one team does much better on hot maps vs cold maps because reasons. And so on.

We haven't even talked about controlling chassis selection, controlling builds, gamemodes, ping, playstyle compatibility -or are these two teams actual competitive teams? which is 100% flawed from the get go by the way.

I'm going to stop here because someone with half a brain should start to see it by now. You use large randomized testing when you can't possibly hope to control every variable. Because once your dataset gets large enough all the random things become background noise to the overall aggregate trend.

tl;dr OP is wrong. As usual when it comes to the "throw out the tests PGI has done because I don't want to believe the results!" crowd.

Edited by Hoax415, 24 September 2014 - 08:51 AM.

[Asyres]

Mcgral18, on 24 September 2014 - 08:35 AM, said:

A test, with controls in place, with 24 competent players, would be the best method.

Switch the team factions after whichever set, and it should get some more accurate data.

That isn't how data works, though. You'd get more accurate data, sure, but the necessarily small sample size would mean that your margin for error would be enormous. I'd argue it would actually be less meaningful than the "all in" testing we've had thus far.

That's not to say that PGI's methodology has been stellar, just that what you're proposing would, at best, be no better.

[Flyby215]

I tend to take issue when I'm solo dropping and the opposing team has 3 Dires, 3 TWs, 3 SCs, and 3 Embers vs my team of a wandering DDC, awesome, trial victor, trial orion, summoner, my Jester... when the inevitable stomp occurs I tend to question balance... be it the mechs or matchmaker I feel kind of ripped off.

Now granted, this doesnt always occur (and not in that exact config) but I hope you see what I'm getting at. Stigma or not, somewhere (mechs or matchmaker) the solo queue should not be so lopsided!

(On my phone atm but when I get home I do have screenshots if anyone wants to call me on it...)

To this end I still feel certain clan mechs can be toned down... but we're close I admit!

[Mcgral18]

Hoax415, on 24 September 2014 - 08:46 AM, said:

No a test with very few players is not a better dataset. You people do not actually understand testing and how it works and how you get robust useful data.

I guess I will try to explain even though its a waste of time with people like Mcral18 who are obviously just pushing an agenda with no basis in reality and its a double waste of time because someone else is just going to start another one of these idiotic threads with the same bad thesis in a couple of days.

Lets say you run your 24 player test. You get two teams of players who are all within 100 elo of eachother per the MM. You have them play 10 matches. 5 where team A is clan and 5 where team B is IS.

According to the logic here that is some kind of super test. Those 10 matches are much more valuable data compared to 200 matches with trial mechs, bads, bad builds, bad chassis, elo difference etc. etc.

Right?

Wrong.

Less players and less matches is not going to get you a better set of data. That's not how it works.

Will you also be controlling for maps? Just run the test on one map right? No wait what if one team is much much better on that map than the other team.

Ok how about we up the test to 16 matches, 4 matches on 4 maps each team plays as each faction twice?

Well we still have the problem of what if one team is stronger on one type of map and you run 3 of those type. Or one team does much better on hot maps vs cold maps because reasons. And so on.

We haven't even talked about controlling chassis selection, controlling builds, gamemodes, ping, playstyle compatibility -or are these two teams actual competitive teams? which is 100% flawed from the get go by the way.

I'm going to stop here because someone with half a brain should start to see it by now. You use large randomized testing when you can't possibly hope to control every variable. Because once your dataset gets large enough all the random things become background noise to the overall aggregate trend.

tl;dr OP is wrong. As usual when it comes to the "throw out the tests PGI has done because I don't want to believe the results!" crowd.

Asyres, on 24 September 2014 - 08:47 AM, said:

That isn't how data works, though. You'd get more accurate data, sure, but the necessarily small sample size would mean that your margin for error would be enormous. I'd argue it would actually be less meaningful than the "all in" testing we've had thus far.

That's not to say that PGI's methodology has been stellar, just that what you're proposing would, at best, be no better.

So, throwing bads such as these:

(Note the 4 trials, 4 sub optimal mechs, and a WubShee)

into a match is what you consider good testing?

You expect those types of players to get accurate results? Notice the crashing there?
Get 100 competent players and do the same thing, it's a better result than PUGs.

Edited by Mcgral18, 24 September 2014 - 09:04 AM.

[Darian DelFord]

At least so far we are all in agreement that the testing done so far is inadequate. Also any ideas that I proposed in the OP are just that ideas that I thought up as I was writing this. I do not pretend to be an experimenting testing genius.But before this 73% is taken to heart, which I think is already to late, you have to run a testing for a longer period of time. Hence the weekend event or something, Put some controls in so not only do you get the raw data (Solo PUG) but also a data set that you can control to an extent.

I was one player in a Kit Fox who just knew how to play the game and how to win it. It does not make my Mech OP or superior. As I stated I wish I was in my damn Jenner, would have been a hell of alot easier for me.

Edited by Darian DelFord, 24 September 2014 - 08:57 AM.

[KingCobra]

Here is the truth there is no IS assault that stacks up against a direwhale there is no IS mech that will stack up to a timberwolf those 2 mechs are the elite Clan mechs and they should be. But no clan mechs should be nerfed what PGI is doing is the right thing by Tweeking the IS mechs to see if they can become more competitive overall.

But the biggest factor in the solo-casual MM battles is skill on both sides Clan and InnerSphere pilots either one can come out on top depending on which one has better overall MWO skills.And I might add sometimes its just dang luck that you get a kill also being less skilled but more lucky.

Also the problem with tournaments is there is no way to drop 1v1 really is still a collection of 2-12 mans bundled together to make a big 12v12 free for all so unless 2 opponents walk over to a corner of a map and go at it without others influencing the outcome and the pilots are of similar skill level its hardtop do controlled tests.

And like I said after playing over 500,000 battles of MechWarrior (MW2-MWO) sometimes it just all boils down to luck in a solo 1v1 or a team 12v12 match.

So in reality PGI and staff can only balance the mechs so far and if they over balance its just as bad as not balancing at all.

[Project_Mercy]

As mentioned, the best way to do it is incentivize people to win. Just take a few and contrast them against their counter parts. Have someone play 30 games in a Griffon, then 30 games in a Stormcrow. 30 games in a Cataphract or Orion and 30 games in a Timberwolf.

I can tell you though, as a 'bad', I basically double my damage in the clan mechs, purely because they're tougher, faster, have better heat management. At least from stormcrow up. That isn't to say there aren't a lot of advantages for IS, it's just that as a whole, they're not all that well balanced. At least the mediums up. I don't cry tears about it though.

Edited by Wraeththix Constantine, 24 September 2014 - 09:03 AM.

[Brody319]

DarthRevis, on 24 September 2014 - 08:45 AM, said:

Tournament, NOOBS, Trial mechs, none of this is valid for testing.

Before we even start we need to form a testable hypothesis in order to do that we must make an educated guess that we can either prove or disprove. and not just a well screw it i think this one...no shots in the dark. Make a REAL educated gue

Just throwing test servers up and having anyone jump in weather the have an agenda or not is ok but it really isn't accurate. Its defiantly a place to start BUT WAS NOT the place PGI should have ended.

I saw no control, no measure of the testable experiments other then Win and Loss which is not always a good indicator (if matches went to time limits or was even deaths at the end or close to) and how many were alive. I never even saw a statement PGI claimed to disprove. Everyone went in with the mindset CLANS ARE OP and im sure the IS guys being bias had something to do with it.


Either way it was poorly executed, poorly researched and just a shot in the dark at something they didn't understand fully.

Let me clarify. I'm not saying just give some clan mechs as trials and take a slice of those matches. I'm saying give new players trial clans and just keep an eye on them over a month. Skill is always a factor, but if clan mechs are op, giving them to new players would result in an average spike in damage, kills, and assists from them.

Skilled players can make any mech good or even great, and this game needs new players to stay alive. If the newer players are being forced into IS mechs and then getting destroyed by Clan mechs they will call "P2W" and leave.

I'm also pretty sure just the weapons is a factor. so we will have to look at how clan mechs do when restricted to only specific groups of weapons. Have a match of players run only missiles, only lasers, and only ballistics and see if any of those cause a large noticeable spike in damage and kills.

[Bacl]

Darian DelFord, on 24 September 2014 - 08:30 AM, said:

The Reason Clan 73% Win Rate On The Last Test Must Be Taken With A Grain of Salt!

First and foremost I am an IS Pilot. I am part of a Davion unit and as such will not be using the clan mechsin CW. With that said I have bought every package that PGI has put out and I will buy the Man of War package as well next pay check. I have a problem with the 73% number that is being thrown around. One of the problems with balance in this game is it is not based on consistency. Meaning Knee Jerk reactions to try to fix a few problem chassis, and as such there are blanket wide changes that truly destroy some mechs and builds.

What I would love to see is another IS vs Clan test using a control group. That control group being players of equal ELO (Prefer Skill base but hard to get those numbers) preferably from the higher ELO range. The reasoning is when you have data coming in from all over the place, less than reliable does not even describe it. In the matches that I participated in during the last test the IS won decidedly in all of them, it was not even a match, it was a ROFLSTOMP by the IS. My question is, what skill levels participated? More importantly, was it consistent players?

Was player A in the same mech throughout the test or did he change into a mech he does not normally play? There are so many variables you cannot base balance based off of a mere percentage which can easily be proven false.

Case in Point, the other morning did a drop just before work, went to make a cup of coffee came back and the game was 3 minutes in and thought damn. I piloted a Kit Fox Prime ECM and 2 LPL’s . By the 3 minute mark the score was 2 to 8 in the enemies favor. By the 5 minute mark it was me alone against 8 other mechs. I thought OK time to hunker down. The map was HPG I was up top so I started sniping like mad, down went an Atlas D. Had to jump down as I was taking fire up top from another ECM mech, went to the basement, ran into an ECM Kitfox with moderate damage, legged then killed him. Scuttled from the basement trying to get to the outside where I ran into an awesome LRM boat with only a MPL as a backup weapon, leg humped him and killed him. Ran into another ECM Kitfox with no damage, managed to get in behind him before he saw me and got two alphas onto a leg and legged him prior to him turning around, he died.

Then a Dual 20 Jager, then a ShadowHawk, then another Jager and finally the DireWolf whom I am assuming had Seismic Sensor as he was perfectly placed every time I tried to get behind him. I toggled my ECM yet again to try to not give away my position with my bubble as I did with the other mechs, but did not work, he backed against a wall. Then with less than 30 seconds remaining he started to move to take me out. Needless to say it came down to the last second and literally at the last second I legged him then took out his other leg and we won the match.

The moral of the story is my Kit Fox is not OP, no one can call it that. It was the player behind the mech that kept his cool knew his own mech’s weakness and strengths as well as the enemies and knew when to strike and when to run. This is why that 73% to me is not reliable. I consider myself an average player in the Kit Fox, the whole time I was doing this, I was wishing I was in my Jenner and how much easier this would have been if I were. I do not consider myself the greatest pilot, I just had a great match. But the calls of the OP’ness of my play by the other team got me to thinking about that stupid 73%.

If PGI really wants to see where the balance level is, hold another tournament but make it IS vs Clans. Or make it a weekend event or even for 24 hours. Announce it, Get more data, before any more nerfs kill the clans. I already laugh at most of them. There just needs to be a better sampling of data then a few hours of PUG match ups in which you match players against other players unevenly.

Man i dont know whats in your coffee but hell i want some!

On a more serious note i think the major issue with this are the inexperienced players that are bound to IS mechs, there is also the veterans that are simply bad pilots ( i rock in stalkers but dont put me in a Atlas....).

Last reason i can think of is the tactic, IS need to charge the clans to negate their range. Forest colony IS vs Clan IS won because an Atlas and Banshee spearheaded into the clans position, fact they didnt last long but the entire team was behind them and it ended up being 12/4 for the IS. That was a spectacular victory. Most Pug game are all about trench warfare and clan will have the advantage on this until the IS team is soft enough for them to charge in.

For these test a button in the match type should be present, "Clan vs IS" so players willing to be part of it can do it on purpose while those who arent confident or too green to do so can avoid it.

[orcrist86]

I'm just going to their this out there. Match maker accounts for elo.... So there was no control grout necessary. The samples are not Ransome they are direct and comparable... So yeah. 73% is probably right on.

[Hoax415]

Mcgral18, on 24 September 2014 - 08:53 AM, said:

see this one game! throw out all the data because this game was bad!

That is why you need to get 100's of matches. The more matches you have the less the X% of them which were predetermined because of multiple afk's or a TK outbreak or one side has all horrible builds matter.

Even with a very small sample in the 73% test. You would need to believe that 20% of solo queue IS vs Clan matches had invalid results but not just invalid, invalid in favor of the clans.

You must believe that 1 in 5 matches were invalid because the Clans couldn't lose and that 0 matches ended up in a way that IS couldn't lose.

Otherwise you are just talking out of your ass as usual.

Edited by Hoax415, 24 September 2014 - 09:09 AM.

[Darian DelFord]

orcrist86, on 24 September 2014 - 09:04 AM, said:

I'm just going to their this out there. Match maker accounts for elo.... So there was no control grout necessary. The samples are not Ransome they are direct and comparable... So yeah. 73% is probably right on.

But the counter to that is, ELO does not equal skill, not by a long shot. My match was an exception. Usually, not always, but usually, one mech can not carry a match. IMHO

[Mcgral18]

Hoax415, on 24 September 2014 - 09:08 AM, said:

That is why you need to get 100's of matches. The more matches you have the less the X% of them which were predetermined because of multiple afk's or a TK outbreak or one side has all horrible builds matter.

Even with a very small sample in the 73% test. You would need to believe that 20% of solo queue IS vs Clan matches had invalid results but not just invalid, invalid in favor of the clans.

You must believe that 1 in 5 matches were invalid because the Clans couldn't lose and that 0 matches ended up in a way that IS couldn't lose.

Otherwise you are just talking out of your ass as usual.

So you're telling me a completely random group of people, using bad, sub optimal, and troll builds, people crashing, and throwing matches, will get better results than 24/48/72/96/etc... competent people who are using effective builds, not throwing matches, and then switching so pilot skill isn't as much a factor.

The underhive is strong in the PUG queue, it doesn't make for good testing. The fact you think it should be used for balance is laughable. AC2 nerf comes to mind.

[wanderer]

You've still got the best Clan 'Mechs behind a paywall when the last Clan vs IS test came out.

While there's plenty of bad pay-pilots, there are inordinately more bad F2P ones. Right now, that means your odds of having a bad player in a heavy/assault slot for the IS is considerably higher, and thus the odds of having one of those dreaded under-100 damage team-mates that can wreck a game singlehandedly.

I'd point more towards light/medium slots being F2P-open now for Clans as the reason for 73% vs. 90% W/L...because the same bads you see in IS 'Mechs are finally being allowed to migrate into Clan chassis, meaning the same reasons for horrible 12-0 style stomps are beginning to get their chance to do so in even better machines to do badly in.